Vehicle camera alignment system

ABSTRACT

A vision system for a vehicle includes at least one imaging sensor disposed at the vehicle and having an exterior field of view. The imaging sensor is operable to capture image data. The imaging sensor includes or is associated with an inclination sensor. At least one other inclination sensor is disposed at the vehicle. A processing system is operable to process outputs of the inclination sensors to determine an alignment or misalignment of the at least one imaging sensor at the vehicle.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a 371 national phase filing of PCTApplication No. PCT/US2012/048880, filed Jul. 30, 2012, which claims thefiling benefits of U.S. provisional application Ser. No. 61/513,745,filed Aug. 1, 2011, which is hereby incorporated herein by reference inits entirety.

FIELD OF THE INVENTION

The present invention relates to imaging systems or vision systems forvehicles and, more particularly, to a vision system that includes atleast one imaging device or camera for capturing images exteriorly ofthe vehicle.

BACKGROUND OF THE INVENTION

Use of imaging sensors in vehicle imaging systems is common and known.Examples of such known systems are described in U.S. Pat. Nos.5,949,331; 5,670,935; and/or 5,550,677, which are hereby incorporatedherein by reference in their entireties. Vehicle vision camera servicecalibration algorithms are common and known. Examples of such knownsystems are described in PCT Publication No. WO 2011/085489, which ishereby incorporated herein by reference in its entirety.

SUMMARY OF THE INVENTION

The present invention provides a vision system or imaging system for avehicle that utilizes one or more cameras to capture images exterior ofthe vehicle, and provides for alignment or alignment adjustment of thecamera or cameras to correct deviations in the camera's alignment overthe life of the vision system. Optionally, the alignment of otherenvironmental sensors for machine vision may be determined and adjustedas well.

The camera or cameras or machine vision sensor or sensors of the vehiclemay include an inclination sensor, such as a three axis inclinationsensor, to detect the orientation of the camera relative to the earth.An output of the inclination sensor may be compared to an output ofanother inclination sensor of the vehicle or of one or more othercameras on the vehicle, and the system may determine when the camera maybe misaligned, whereby the camera may be adjusted towards its initial ortargeted orientation or alignment.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view of a vehicle with a vision system and imagingsensors or cameras that provide exterior fields of view in accordancewith the present invention;

FIG. 2 is a schematic of a camera and inclination sensor in accordancewith the present invention;

FIG. 3 is a perspective view of a target suitable for use in calibratinga camera for a vehicle;

FIG. 4 is a perspective view of an image captured by a camera of avehicle;

FIG. 5 is a side elevation and schematic of a vehicle with a visionsystem and cameras in accordance with the present invention;

FIG. 6 is an illustration of a 360 degree top vision view as captured bythe vision system and cameras of the present invention;

FIG. 7 is a view of captured images with two cameras well aligned andtwo cameras misaligned;

FIG. 8A is a schematic showing a comparison of pixel resolution and atypical inclination sensor accuracy;

FIG. 8B is a schematic showing how the steradian angle θ adds up from ahorizontal component θ_(b) and a vertical component θ_(c), with (b) and(c) being the resulting opposite legs of θ_(b) and θ_(c), and (a) beingthe geometrical sum of (b) plus (c) and the resulting opposite leg of θ(θ_(c) equates to a in FIG. 8A);

FIG. 9 is an example of a plan view of a vehicle with a combined humanand machine vision system and five cameras and two radar sensors thatprovide exterior fields of view in accordance with the presentinvention;

FIG. 10 is a schematic of the system of FIG. 9, showing the visionsystem's architecture in accordance with the present invention;

FIG. 11 is a flow chart showing an initialization and alignmentcorrection algorithm in accordance with the present invention;

FIG. 12 is a schematic showing a surround view system architecturescheme having camera alignment sensors within some or all cameras inaccordance with the present invention;

FIG. 13 is a schematic showing a surround view system architecturescheme having camera alignment sensors within some or all cameras and anadditional inclination sensor within the ECU in accordance with thepresent invention;

FIG. 14 is a schematic showing a surround view system architecturescheme having camera alignment sensors within some or all cameras and anadditional inclination sensor attached to the vehicle in accordance withthe present invention; and

FIGS. 15A-D are tables showing the relation between a misalignment angleand a resulting deviation in distance.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings and the illustrative embodiments depictedtherein, a vehicle 10 includes an imaging system or vision system 12that includes at least one imaging sensor or camera 14 (such as fourimaging sensors or cameras 14 a, 14 b, 14 c, 14 d at the front (or atthe windshield), rear and sides of the vehicle, respectively), whichcapture images exterior of the vehicle (FIG. 1). The camera 14 includesan inclination sensor 16 (FIG. 2), such as a three axis inclinationsensor or the like, which is operable to determine or detect or sensethe orientation of the camera relative to the earth (such as the tilt orpitch or yaw or levelness of the camera). The vision system 12 isoperable to compare an output of the inclination sensor 16 to an outputof another inclination sensor of the vehicle or another inclinationsensor or sensors of one or more of the other cameras mounted at thevehicle, as discussed below. Responsive to such comparison, the visionsystem 12 may determine when one or more of the cameras becomesmisaligned, whereby the system may operate to adjust or reorient thecamera towards its initial or targeted orientation or alignment orcalibrated state, as also discussed below.

Camera based systems (such as multi-camera systems like a surround viewsystem or the like, or a rear view camera or the like) typically requireprocessing to correct deviations in the pitch, yaw and/or roll of thecamera (extrinsic calibrations) over the life-time of the system and/orthe vehicle, such as due to temperature variations or positionalshifting of the camera due to vibrations and/or the like. Suchprocessing may be required in order to provide proper image stitching ormerging of images captured by multiple cameras (such as for a panoramicrearward view or such as for a surround view system or top-down orbirds-eye view or the like), and/or to provide proper overlaypositioning on the displayed images, such as for rear backup assistsystems and the like.

Typically, automotive cameras are calibrated offline (without driving ofthe vehicle, such as in the vehicle assembly or production line), usingtargets, such as the target 18 shown in FIG. 3, where the positions andorientation of the target lines relative to the camera and vehicle areknown. Optionally, some automotive cameras use image processing (such asof images or image data as shown in FIG. 4) to detect and correct formisalignment of the camera after the camera is installed at the vehicleand while the vehicle is driven on the road. For example, such alignmentdetection and correction via image processing is described in U.S. Pat.Nos. 7,991,522; 7,877,175 and 7,720,580, and/or R. Hartley and A.Zisserman, “Multiple View Geometry in Computer Vision”, Cambridge, ISBN978-0-521-54051-3, which are all hereby incorporated herein by referencein their entireties. Such image processing techniques work well fortheir intended purposes, but may require additional processing basedonline (during driving) calibration algorithms, and thus may requireadditional processing power (such as tracking, vanishing pointestimation, trajectory calculations and the like).

The present invention provides an inclination sensor at the camera orcameras or environmental machine vision sensor or sensors, such as, forexample, a radar sensor or a lidar sensor or the like, to determine achange in the position or orientation or alignment of the camera ormachine vision sensor(s) over the life of the vehicle and/or visionsystem and/or camera or machine vision sensor(s). If the inclinationsensor is installed in the camera (or is otherwise fixedly associatedwith the camera) and the relation between the camera and the inclinationsensor is known, the orientation of the camera to the earth can bereadily measured or determined without further image processing. Forexample, the inclination sensor may comprise a three axis inclinationsensor, and the sensor's roll, pitch and yaw may be readily obtained bythe vision system circuitry or control, without requiring furthercomplex algorithms and image processing. Because the vehicle-to-earthrelationship is not fixed (the vehicle travels on different roads orsurfaces with different tilts, slopes, etc.), the system of the presentinvention includes a second inclination sensor or reference inclinationsensor at the vehicle so that the system can calibrate the cameracorrectly to the vehicle at which it is mounted and not to a particulargeographic location at the earth where the vehicle may be located.

For example, and with reference to FIG. 2, an inclination sensor, suchas a three-axis inclination sensor 16 of a camera 14, is operable todetect changes in the yaw, pitch and roll of the sensor and camera. Theknown relation between the camera and sensor allows for readilydetecting changes for each angle (yaw, pitch and roll). With referenceto FIG. 2, the angle β is known due to the relation of the inclinationsensor to the camera (and may be fixed), and thus a change of the pitch(for example) of the camera may be determined or measured responsive tothe inclination sensor output (where the pitch is determined as Y=β−α).Such inclination sensors provide high accuracy and resolution, and havelong-run stability (such as, for example, a deviation of less than about0.014 degrees after more than 500 hours in operation), and provide highreproducibility (such as better than about 0.01 degrees). Suchinclination sensors are suitable for use in automotive applications andare currently used in other automotive applications.

The example in FIG. 5 demonstrates how little the effect of an exemplarydeviation of about 0.014 degrees (plus 0.01 degrees tolerance equalsabout 0.024 degrees) is in a typical rear camera range of interest ofabout 5 and 7 meters turns out. In this example a describes a deviationangle between the camera's calibration direction and the camera's realviewing direction. The real viewing direction becomes measured by theinclination sensor 16 b assembled to camera 14 b (another sensor 16 c isassembled or disposed at a side mirror camera 14 c and another sensor 16a is disposed at camera 14 a attached to vehicle 10). In the (rear)distance of seven meters, the deviation is just at 2.93 mm, which ismuch less than the typical resolution a three megapixel image sensor(1920×1600 pixel) has in it's center region (highest resolution area)for a rear view camera with fish eye optics having about an openingangle of about 185 degrees. Often the pixel resolution is already in the‘cm’ area or thereabouts at that distance.

In virtual top view vision systems, often a radius of maximal about 5 mis visible around a vehicle. FIG. 6 shows an illustration of a 360degrees top vision view. These top views are generated by stitching,undistorting and dewarping of transmorphed views of single cameras toone combined view. There are stitching edges between the single cameras'borderlines. In cases where the cameras are not aligned (calibrated)well, distortion between the images become visible, especially whenstraight lines on the ground around the vehicle are across a stitchingedge. FIG. 7 shows a real scene image with two cameras that are wellaligned (upper left stitching area) and two cameras that are misaligned(lower right stitching area). The left out center region of the image ofFIG. 7 is where the vehicle actually is, because the ground under thevehicle is not captured by any camera of the system. When themisalignment is less than one pixel dot size, the distortion becomessubstantially invisible or not discernible. In the shown example of FIG.7, 1280×800 pixel (SVGA) cameras were in use. The resulting pixel sizewithin a stitched top view is at about 2 cm.

FIG. 15B is a table that shows the relation between the misalignmentangle and the resulting deviation in distance. The relation between thedeviation in the given distance, the misalignment angle and thedeviation is given by the geometrical mathematics: the opposing leg(deviation in the given distance) divided by the adjacent leg (distance)equates to the tangent misalignment angle. To receive the angle thearcos tangent of the division has to be used (see FIG. 15C). Since adeviation can be horizontal and vertical (two dimensional) at one time,the deviating angle adds up geometrically to a steradian angle θ (seeFIG. 15D and FIGS. 8A and 8B). As shown in FIG. 8B, the steradian angleθ adds up from a horizontal component θ_(b) and a vertical componentθ_(c), with (b) and (c) being the resulting opposite legs of θ_(b) andθ_(c), respectively, and (a) being the geometrical sum of (b) plus (c)and also being the resulting opposite leg of θ. The angle θ_(c) equatesto α in FIG. 8A (the vertical component of the angle).

The distances 5 m and 7 m are listed in FIG. 15A. It becomes clear thatif the camera's alignment deviation stays under about 0.229 degrees,than no pixel distortion becomes visible within a SVGA top view havingabout a 2 cm pixel dot size (see, for example, FIG. 8A, where the pixelresolution at a 5 m distance of a 1280×800 pixel sensing system of 2 cmis shown as compared to a typical inclination sensor accuracy of about0.014 degrees+0.01 degrees, with the scaling in FIG. 8A beingexaggerated to improve the readability). At megapixel cameras (1920×1600pixel) the pixel dot size is at about 1 cm within a top view image with5 m as visible area around a vehicle. This ratio shows that the accuracyof inclination sensors (of about 0.014 degrees+/−0.01 degrees) is aboutfive times higher in megapixel top view vision systems and about tentimes higher than necessary to calibrate the visions system's camerasaccurately enough to substantially prevent or limit visible distortion(less than 1 pixel).

In practice, the most economic choice would be to pick the leastaccurate sensor in angular accuracy which still allows the system tocalibrate the vision system's cameras accurate enough so that thedeviation at the furthest visible point within a top view screen isstill less than one pixel. Within a top view system, the visible areaaround the car is limited. Typically, the visible area in front andbehind the car is at about 5 meters (limited longitudinal extension) andthe visible area at the side of the car is typically at about 3 meters(limited latitudinal extension), with the longitudinal extension beinggenerally orthogonal to the longitudinal extension (the longitudinalextension typically is the longer one). Points at the far corners arethe furthest. The inclination sensor angular (steradian) accuracy limitis given by the arcos tangent of the pixel size resolution at thefurthest visible point at the screen divided by the distance to thatpoint (see FIG. 15D).

Because tangent alpha is equal to the opposite leg of the triangle ordistance divided by the adjacent leg of the triangle (so the deviationin the distance), the system is able to compare which deviation in thedistance would become captured by one pixel. This is why the longestdistance which is visible within the top view (typically thelongitudinal distance, which is typically around 5 m) is preferablyselected. As long the angular resolution of an inclination sensor ismore accurate than the arcus tangent of the highest possible deviationin distance divided by the distance, the misalignment of a camera doesnot become visible. Since a deviation can occur with mixed vertical andhorizontal components at the same time, the system may utilize thesteradian angle θ in the tangential equation.

A single camera vision system of the present invention (such as a rearvision system having a rearward facing camera at a rear portion of thevehicle) may, for example, have an inclination sensor at the camera anda second inclination sensor at the vehicle (16 d in FIG. 5). The secondinclination sensor may be disposed within the ECU or anywhere at thevehicle (such as, for example, at the head unit of the vehicle or anyother suitable location at the vehicle), as long as its initialorientation relative to the initial orientation of the camera'sinclination sensor is known. During use of the vehicle and visionsystem, the system may compare the outputs of the two inclinationsensors and may thus determine when there is a change in orientation ofthe camera's inclination sensor and thus a change in orientation of thecamera. Responsive to such a detection or determination, the system maygenerate an alert (such as to indicate that a particular camera ismisaligned) or may adjust the camera or image data captured by thecamera (such as via a physical adjustment of the camera orientation orsuch as via image processing or the like) to account for or correct forthe detected or determined misalignment or change in orientation of thecamera.

Optionally, a multi-camera vision system (such as for a surround-viewsystem or the like) may have multiple cameras disposed at the vehiclewith exterior fields of view around the vehicle (such as shown in FIGS.1, 5, 6, 7 and 9), with most of the cameras having an inclination sensorassociated therewith or attached thereto (for example, and as shown inFIGS. 9 and 10, a combined human and machine vision system may compriseor include one or more, such as two, radar sensors in addition to a fouror five camera system). In such an application, an additionalinclination sensor is not needed at the vehicle but such a sensor isoptional as an additional reference (see reference 16 d in FIG. 5 andsee FIG. 14). The initial outputs of the cameras when properly orientedand aligned and calibrated will be received and/or input and/or known bythe vision system control. Thus, the vision system control, knowing theinitial outputs of the cameras' inclination sensors and thus the initialrelative orientations of the cameras, can monitor the outputs of theinclination sensors and determine when one of them changes relative tothe others. Responsive to such a detection or determination, the controlmay generate an alert (such as to indicate that a particular camera ismisaligned) or may adjust the physical orientation of the misalignedcamera or adjust the image processing of the image data captured by themisaligned camera or the like, in order to account for the misalignment.The multi-camera vision system thus may determine misalignment of acamera at the vehicle via the inclinometers at the cameras and anadditional algorithm. The algorithm may process the sensor outputs andmay determine, responsive to a relative change in the outputs of thesensors (that would be indicative of the orientation of one of thesensors changing relative to the vehicle and that would not be merelyindicative of the vehicle orientation changing), which sensor or sensorsis/are the reference sensor/sensors and which sensor is misaligned. Thesystem may determine which of the sensors to use as the reference andcompare the outputs of the other sensors to the reference sensor todetermine the alignment or misalignment.

As shown in FIG. 9, a vehicle 10 includes or has a combined human andmachine vision system, which includes five high dynamic range (HDR)cameras, such as a front right side corner camera 14 a′, a front leftside corner camera 14 a″, a rear or rearward facing center camera 14 band wide angle left and right side cameras 14 c, 14 d that provideexterior fields of view in accordance with the present invention(optionally, the system may include a front or forward facing centercamera 14 a). The system also includes a front radar sensor 140 and arear radar sensor 141. FIG. 10 is a schematic of the specific example ofFIG. 9 showing the vision system's architecture in accordance with thepresent invention. The Oculus Module may comprise a system-on-chip orElectronic Control Unit (ECU). In the illustrated embodiment, the LVDSdata line comprises a full duplex. The front facing camera may comprisean EPM3™ camera or the like, and may have or utilize or be associatedwith an image processor, such as am EyeQ™ or EyeQ2™ or EyeQ3™ processor,such as are provided by Mobileye® (where the image processor utilizesobject detection software or the like in processing image data capturedby the camera or cameras), or the like. The image data captured by thefive cameras (such as five HDR cameras) may be communicated to themodule via a video bus communication link 2, and the module may providea control signal 5 to the cameras. A data bus (such as a CAN or LIN orEthernet data bus or network or an HSCAN data bus or network) may beutilized to communicate signals between the module and the radar sensors(and optionally the front facing camera) and/or the vehicle. The systemmay generate control signals that provide braking control and/or vehiclesteering control responsive to the captured image data and outputs ofthe radar sensors.

An exemplary initialization and alignment correction algorithm accordingto the present invention is shown in the chart of FIG. 11. An accordingsurround view system architecture scheme having camera alignment sensors(inclinometers) within some or all of the cameras is shown in the chartof FIG. 12. FIG. 13 is a chart showing a preferred embodiment of thepresent invention's system architecture having an additional inclinationsensor assembled within the ECU for providing a reference normal signalin addition to inclination sensor signals of the visions system'scameras. FIG. 14 is a chart showing an example of the architecturehaving the additional inclination sensor attached to the vehicle (seereference 16 d in FIG. 5) providing a (vehicle's level) reference normalsignal in addition to inclination sensor signals of the visions system'scamera via CAN or LIN bus interfaces (or other suitable communicationnetwork or link or Ethernet communication link or the like) to thevision system control or MCU.

The camera alignment correction/calibration algorithm may not always beallowed to run due to manufacturer requirements. For example, there maybe limited modes in which a calibration run is enabled. Also, theremight be limitations in the minimum required start up time or shut downtime or to energy consumption while idling. The transfer channels mighthave limited capacity as well. To cope with these matters, the transferof the three axis (vector) of an inclinometer may be transferred withinthe SYNC gap between images during run time or incorporate into theimage stream, such as by utilizing aspects of the systems described inU.S. provisional application Ser. No. 61/567,446, filed Dec. 6, 2011,which is hereby incorporated herein by reference in its entirety. Theinclination vector data may be stored in the MCU so as to be present attimes when a calibration cycle is enabled.

The vision system may include a display screen that is disposed in thevehicle (such as at an interior rearview mirror assembly or windshieldelectronics module or the like in the cabin of the vehicle and at ornear the in-cabin surface of the vehicle windshield) and viewable by thedriver of the vehicle. The vision system is operable to display imagesrepresentative of captured image data on the display screen. The visionsystem is operable in a virtual top view vision mode and, when operatingin the top view vision mode, the processing system processes image datacaptured by the imaging sensors to at least one of combine andmanipulate the image data, and, when operating in the top view visionmode, the vision system displays an exterior scene (such as via thecombination and/or manipulation or stitching or merging of the imagedata captured by the plurality of cameras) around the vehicle. Thedisplayed exterior scene is around the vehicle and may have a firstlimited extension in one dimension (such as forwardly of the vehicle andalong the field of view of the forward facing camera) and a secondlimited extension in a second orthogonal dimension (such as sidewardlyof the vehicle), wherein the result of a tangent steradian accuracy ofthe at least one inclination sensor divided by a longer one of the firstand second limited extensions is greater than the highest resolution ofat least one of the image sensors along the longer limited extension'spath.

Thus, the present invention provides a means for determining amisalignment of a camera at a vehicle. If the intrinsic parameters of acamera are measured together with the inclinometer of the camera (suchas inside the camera or at the camera), the camera may be aligned to thevehicle without any target in the camera's field of view and without anyimage processing of images captured by the camera. The changes of thecamera-to-vehicle alignment over the vehicle or camera or system life ordue to temperature variations or the like can be readily corrected usingrelatively simple algorithms (either while the vehicle is being drivenon a road or when the vehicle is parked or otherwise not in use). Also,the system of the present invention allows for replacement of a damagedcamera without requiring special service calibration procedures (forexample, no drives or targets are needed to calibrate/align the newcamera). Also, the present invention does not require any complicatedpreconditions when using image processing algorithms (dependencies toother software modules such as a camera control function or the like).

The system of the present invention thus utilizes two or moreinclination sensors or level sensors, and the two or more inclinationsensors may be part of or associated with respective cameras, or one ormore inclination sensors may be associated with a respective one or morecameras and another inclination sensor may be disposed at the vehicleseparate from any camera. For example, the system may utilize aninclination sensor that is already part of or incorporated at thevehicle, such as a level sensor or the like that is used by a levelingsystem of the vehicle to adjust the suspension to level the vehicle. Thesystem utilizes two inclination sensors to detect a misalignment orthreshold degree of misalignment of one or more cameras (having aninclination sensor associated therewith) mounted at the vehicle, such asat an exterior rearview mirror assembly of the vehicle and/or at a rearportion of the vehicle and/or at a front portion of the vehicle and/orat an interior rearview mirror assembly and/or windshield electronicsmodule of the vehicle. The system of the present invention may beparticularly useful when the camera and inclination sensor orinclinometer are disposed or used in an exterior rearview or sideviewmirror assembly, and the system of the present invention may utilizeaspects of (or may operate in conjunction with or separate from) thesystems described in U.S. Pat. Nos. 8,066,415; 7,991,522; and/or7,720,580, and/or PCT Application No. PCT/CA2012/000378, filed Apr. 25,2012, and/or U.S. patent application Ser. No. 12/508,840, filed Jul. 24,2009 and published Jan. 28, 2010 as U.S. Publication No.US-2010-0020170, and/or U.S. provisional applications, Ser. No.61/570,017, filed Dec. 13, 2011; Ser. No. 61/588,833, filed Jan. 20,2012; and/or Ser. No. 61/613,651, filed Mar. 21, 2012, which are allhereby incorporated herein by reference in their entireties.

The imaging sensor and its photosensor array may comprise any suitablecamera or sensing device, such as, for example, an array of a pluralityof photosensor elements arranged in 640 columns and 480 rows (a 640×480imaging array) or more (such as discussed above), with a respective lensfocusing images onto respective portions of the array. The photosensorarray may comprise a plurality of photosensor elements arranged in aphotosensor array having rows and columns.

The logic and control circuit of the imaging sensor may function in anyknown manner, such as in the manner described in U.S. Pat. Nos.5,550,677; 5,877,897; 6,498,620; 5,670,935; 5,796,094; 6,396,397, and/orU.S. patent application Ser. No. 13/534,657, filed Jun. 27, 2012, and/orU.S. provisional applications, Ser. No. 61/666,146, filed Jun. 29, 2012;Ser. No. 61/653,665, filed May 31, 2012; Ser. No. 61/653,664, filed May31, 2012; Ser. No. 61/650,667, filed May 23, 2012; Ser. No. 61/624,507,filed Apr. 16, 2012; Ser. No. 61/616,126, filed Mar. 27, 2012; Ser. No.61/615,410, filed Mar. 26, 2012; Ser. No. 61/613,651, filed Mar. 21,2012; Ser. No. 61/607,229, filed Mar. 6, 2012; Ser. No. 61/605,409,filed Mar. 1, 2012; Ser. No. 61/602,878, filed Feb. 24, 2012; Ser. No.61/602,876, filed Feb. 24, 2012; Ser. No. 61/600,205, filed Feb. 17,2012; Ser. No. 61/588,833, filed Jan. 20, 2012; Ser. No. 61/583,381,filed Jan. 5, 2012; Ser. No. 61/579,682, filed Dec. 23, 2011; Ser. No.61/570,017, filed Dec. 13, 2011; Ser. No. 61/568,791, filed Dec. 9,2011; Ser. No. 61/567,446, filed Dec. 6, 2011; Ser. No. 61/559,970,filed Nov. 15, 2011; Ser. No. 61/552,167, filed Oct. 27, 2011; Ser. No.61/540,256, filed Sep. 28, 2011; and/or Ser. No. 61/511,738, filed Jul.26, 2011, which are all hereby incorporated herein by reference in theirentireties. The system may communicate with other communication systemsvia any suitable means, such as by utilizing aspects of the systemsdescribed in PCT Application No. PCT/US10/038477, filed Jun. 14, 2010,and/or U.S. patent application Ser. No. 13/202,005, filed Aug. 17, 2011,now U.S. Pat. No. 9,126,525, and/or U.S. provisional applications, Ser.No. 61/567,150, filed Dec. 6, 2011; Ser. No. 61/565,713, filed Dec. 1,2011; and/or Ser. No. 61/537,279, filed Sep. 21, 2011, which are herebyincorporated herein by reference in their entireties.

The imaging device and control and image processor and any associatedillumination source, if applicable, may comprise any suitablecomponents, and may utilize aspects of the cameras and vision systemsdescribed in U.S. Pat. Nos. 5,550,677; 5,877,897; 6,498,620; 5,670,935;5,796,094; 6,396,397; 6,806,452; 6,690,268; 7,005,974; 7,123,168;7,004,606; 6,946,978; 7,038,577; 6,353,392; 6,320,176; 6,313,454; and6,824,281, and/or International Publication No. WO 2010/099416,published Sep. 2, 2010, and/or PCT Application No. PCT/US10/47256, filedAug. 31, 2010, and/or PCT Application No. PCT/US2012/048110, filed Jul.25, 2012, and/or U.S. patent application Ser. No. 12/508,840, filed Jul.24, 2009, and published Jan. 28, 2010 as U.S. Pat. Publication No. US2010-0020170, and/or U.S. patent application Ser. No. 13/534,657, filedJun. 27, 2012 and published Jan. 3, 2013 as U.S. Publication No.US-2013-0002873, which are all hereby incorporated herein by referencein their entireties. The camera or cameras may comprise any suitablecameras or imaging sensors or camera modules, and may utilize aspects ofthe cameras or sensors described in U.S. patent application Ser. No.12/091,359, filed Apr. 24, 2008 and published Oct. 1, 2009 as U.S.Publication No. US-2009-0244361; and/or Ser. No. 13/260,400, filed Sep.26, 2011, now U.S. Pat. No. 8,542,451, and/or U.S. Pat. Nos. 7,965,336and/or 7,480,149, which are hereby incorporated herein by reference intheir entireties. The imaging array sensor may comprise any suitablesensor, and may utilize various imaging sensors or imaging array sensorsor cameras or the like, such as a CMOS imaging array sensor, a CCDsensor or other sensors or the like, such as the types described in U.S.Pat. Nos. 5,550,677; 5,670,935; 5,760,962; 5,715,093; 5,877,897;6,922,292; 6,757,109; 6,717,610; 6,590,719; 6,201,642; 6,498,620;5,796,094; 6,097,023; 6,320,176; 6,559,435; 6,831,261; 6,806,452;6,396,397; 6,822,563; 6,946,978; 7,339,149; 7,038,577; 7,004,606;7,720,580; and/or 7,965,336, and/or PCT Application No.PCT/US2008/076022, filed Sep. 11, 2008 and published Mar. 19, 2009 asInternational Publication No. WO/2009/036176, and/or PCT Application No.PCT/US2008/078700, filed Oct. 3, 2008 and published Apr. 9, 2009 asInternational Publication No. WO/2009/046268, which are all herebyincorporated herein by reference in their entireties.

The camera module and circuit chip or board and imaging sensor may beimplemented and operated in connection with various vehicularvision-based systems, and/or may be operable utilizing the principles ofsuch other vehicular systems, such as a vehicle headlamp control system,such as the type disclosed in U.S. Pat. Nos. 5,796,094; 6,097,023;6,320,176; 6,559,435; 6,831,261; 7,004,606; 7,339,149; and/or 7,526,103,which are all hereby incorporated herein by reference in theirentireties, a rain sensor, such as the types disclosed in commonlyassigned U.S. Pat. Nos. 6,353,392; 6,313,454; 6,320,176; and/or7,480,149, which are hereby incorporated herein by reference in theirentireties, a vehicle vision system, such as a forwardly, sidewardly orrearwardly directed vehicle vision system utilizing principles disclosedin U.S. Pat. Nos. 5,550,677; 5,670,935; 5,760,962; 5,877,897; 5,949,331;6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202;6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452;6,822,563; 6,891,563; 6,946,978; and/or 7,859,565, which are all herebyincorporated herein by reference in their entireties, a trailer hitchingaid or tow check system, such as the type disclosed in U.S. Pat. No.7,005,974, which is hereby incorporated herein by reference in itsentirety, a reverse or sideward imaging system, such as for a lanechange assistance system or lane departure warning system or for a blindspot or object detection system, such as imaging or detection systems ofthe types disclosed in U.S. Pat. Nos. 7,881,496; 7,720,580; 7,038,577;5,929,786 and/or 5,786,772, and/or U.S. provisional applications, Ser.No. 60/628,709, filed Nov. 17, 2004; Ser. No. 60/614,644, filed Sep. 30,2004; Ser. No. 60/618,686, filed Oct. 14, 2004; Ser. No. 60/638,687,filed Dec. 23, 2004, which are hereby incorporated herein by referencein their entireties, a video device for internal cabin surveillanceand/or video telephone function, such as disclosed in U.S. Pat. Nos.5,760,962; 5,877,897; 6,690,268; and/or 7,370,983, and/or U.S. patentapplication Ser. No. 10/538,724, filed Jun. 13, 2005 and published Mar.9, 2006 as U.S. Publication No. US-2006-0050018, which are herebyincorporated herein by reference in their entireties, a traffic signrecognition system, a system for determining a distance to a leading ortrailing vehicle or object, such as a system utilizing the principlesdisclosed in U.S. Pat. Nos. 6,396,397 and/or 7,123,168, which are herebyincorporated herein by reference in their entireties, and/or the like.

Optionally, the circuit board or chip may include circuitry for theimaging array sensor and or other electronic accessories or features,such as by utilizing compass-on-a-chip or EC driver-on-a-chip technologyand aspects such as described in U.S. Pat. No. 7,255,451 and/or U.S.Pat. No. 7,480,149; and/or U.S. patent application Ser. No. 11/226,628,filed Sep. 14, 2005 and published Mar. 23, 2006 as U.S. Publication No.US-2006-0061008, and/or Ser. No. 12/578,732, filed Oct. 14, 2009 andpublished Apr. 22, 2010 as U.S. Publication No. US-2010-0097469, whichare hereby incorporated herein by reference in their entireties.

Optionally, the vision system may include a display for displayingimages captured by one or more of the imaging sensors for viewing by thedriver of the vehicle while the driver is normally operating thevehicle. Optionally, for example, the vision system may include a videodisplay device disposed at or in the interior rearview mirror assemblyof the vehicle, such as by utilizing aspects of the video mirror displaysystems described in U.S. Pat. No. 6,690,268 and/or U.S. patentapplication Ser. No. 13/333,337, filed Dec. 21, 2011, now U.S. Pat. No.9,264,672, which are hereby incorporated herein by reference in theirentireties. The video mirror display may comprise any suitable devicesand systems and optionally may utilize aspects of the compass displaysystems described in U.S. Pat. Nos. 7,370,983; 7,329,013; 7,308,341;7,289,037; 7,249,860; 7,004,593; 4,546,551; 5,699,044; 4,953,305;5,576,687; 5,632,092; 5,677,851; 5,708,410; 5,737,226; 5,802,727;5,878,370; 6,087,953; 6,173,508; 6,222,460; 6,513,252; and/or 6,642,851,and/or European patent application, published Oct. 11, 2000 underPublication No. EP 0 1043566, and/or U.S. patent application Ser. No.11/226,628, filed Sep. 14, 2005 and published Mar. 23, 2006 as U.S.Publication No. US-2006-0061008, which are all hereby incorporatedherein by reference in their entireties. Optionally, the video mirrordisplay screen or device may be operable to display images captured by arearward viewing camera of the vehicle during a reversing maneuver ofthe vehicle (such as responsive to the vehicle gear actuator beingplaced in a reverse gear position or the like) to assist the driver inbacking up the vehicle, and optionally may be operable to display thecompass heading or directional heading character or icon when thevehicle is not undertaking a reversing maneuver, such as when thevehicle is being driven in a forward direction along a road (such as byutilizing aspects of the display system described in PCT Application No.PCT/US2011/056295, filed Oct. 14, 2011 and published Apr. 19, 2012 asInternational Publication No. WO 2012/051500, which is herebyincorporated herein by reference in its entirety). The display screen ordisplay device may be disposed at or in the interior rearview mirrorassembly or may comprise a navigational display of the vehicle, such asmay be disposed at the instrument panel or console of the vehicle or thelike.

As discussed above, the vision system (utilizing a forward and/orrearward facing camera and other cameras disposed at the vehicle withexterior fields of view) comprises and utilizes a plurality of cameras(such as utilizing a rearward facing camera and sidewardly facingcameras and a forwardly facing camera disposed at the vehicle), andprovides a display of a top-down view or birds-eye view of the vehicleor a surround view at the vehicle, such as by utilizing aspects of thevision systems described in PCT Application No. PCT/US10/25545, filedFeb. 26, 2010 and published on Sep. 2, 2010 as International PublicationNo. WO 2010/099416, and/or PCT Application No. PCT/US10/47256, filedAug. 31, 2010 and published Mar. 10, 2011 as International PublicationNo. WO 2011/028686, and/or PCT Application No. PCT/US11/62834, filedDec. 1, 2011 and published Jun. 7, 2012 as International Publication No.WO 2012-075250, and/or U.S. patent application Ser. No. 13/333,337,filed Dec. 21, 2011, now U.S. Pat. No. 9,264,672, and/or U.S.provisional applications, Ser. No. 61/615,410, filed Mar. 26, 2012; Ser.No. 61/613,651, filed Mar. 21, 2012; Ser. No. 61/588,833, filed Jan. 20,2012; Ser. No. 61/570,017, filed Dec. 13, 2011; Ser. No. 61/568,791,filed Dec. 9, 2011; Ser. No. 61/559,970, filed Nov. 15, 2011; Ser. No.61/540,256, filed Sep. 28, 2011, which are hereby incorporated herein byreference in their entireties.

Optionally, the video mirror display may be disposed rearward of andbehind the reflective element assembly and may comprise a display suchas the types disclosed in U.S. Pat. Nos. 5,530,240; 6,329,925;7,855,755; 7,626,749; 7,581,859; 7,338,177; 7,274,501; 7,255,451;7,195,381; 7,184,190; 5,668,663; 5,724,187 and/or 6,690,268, and/or inU.S. patent application Ser. No. 11/226,628, filed Sep. 14, 2005 andpublished Mar. 23, 2006 as U.S. Publication No. US-2006-0061008; and/orSer. No. 10/538,724, filed Jun. 13, 2005 and published Mar. 9, 2006 asU.S. Publication No. US-2006-0050018, which are all hereby incorporatedherein by reference in their entireties. The display is viewable throughthe reflective element when the display is activated to displayinformation. The display element may be any type of display element,such as a vacuum fluorescent (VF) display element, a light emittingdiode (LED) display element, such as an organic light emitting diode(OLED) or an inorganic light emitting diode, an electroluminescent (EL)display element, a liquid crystal display (LCD) element, a video screendisplay element or backlit thin film transistor (TFT) display element orthe like, and may be operable to display various information (asdiscrete characters, icons or the like, or in a multi-pixel manner) tothe driver of the vehicle, such as passenger side inflatable restraint(PSIR) information, tire pressure status, and/or the like. The mirrorassembly and/or display may utilize aspects described in U.S. Pat. Nos.7,184,190; 7,255,451; 7,446,924 and/or 7,338,177, which are all herebyincorporated herein by reference in their entireties. The thicknessesand materials of the coatings on the substrates of the reflectiveelement may be selected to provide a desired color or tint to the mirrorreflective element, such as a blue colored reflector, such as is knownin the art and such as described in U.S. Pat. Nos. 5,910,854; 6,420,036;and/or 7,274,501, which are hereby incorporated herein by reference intheir entireties.

Optionally, the display or displays and any associated user inputs maybe associated with various accessories or systems, such as, for example,a tire pressure monitoring system or a passenger air bag status or agarage door opening system or a telematics system or any other accessoryor system of the mirror assembly or of the vehicle or of an accessorymodule or console of the vehicle, such as an accessory module or consoleof the types described in U.S. Pat. Nos. 7,289,037; 6,877,888;6,824,281; 6,690,268; 6,672,744; 6,386,742; and 6,124,886, and/or U.S.patent application Ser. No. 10/538,724, filed Jun. 13, 2005 andpublished Mar. 9, 2006 as U.S. Publication No. US-2006-0050018, whichare hereby incorporated herein by reference in their entireties.

The display or displays may comprise a video display and may utilizeaspects of the video display devices or modules described in U.S. Pat.Nos. 6,690,268; 7,184,190; 7,274,501; 7,370,983; 7,446,650; and/or7,855,755, and/or U.S. patent application Ser. No. 10/538,724, filedJun. 13, 2005 and published Mar. 9, 2006 as U.S. Publication No.US-2006-0050018, which are all hereby incorporated herein by referencein their entireties. The video display may be operable to display imagescaptured by one or more imaging sensors or cameras at the vehicle.

Changes and modifications to the specifically described embodiments maybe carried out without departing from the principles of the presentinvention, which is intended to be limited only by the scope of theappended claims as interpreted according to the principles of patentlaw.

The invention claimed is:
 1. A vision system for a vehicle, said visionsystem comprising: a plurality of imaging sensors disposed at thevehicle and having respective exterior fields of view, said imagingsensors capturing respective image data, wherein each of said imagingsensors includes a respective inclination sensor; at least one otherinclination sensor disposed at the vehicle; a processing system that isoperable to process outputs of said inclination sensors to determine analignment or misalignment of said at least one imaging sensor at thevehicle; a display screen in the vehicle and viewable by the driver ofthe vehicle; wherein said vision system is operable to display imagesrepresentative of said captured image data on said display screen;wherein said vision system is operable in a virtual top view visionmode, and wherein, when operating in said virtual top view vision mode,said processing system processes image data captured by said pluralityof imaging sensors to at least one of combine and manipulate said imagedata, and wherein, when operating in said virtual top view vision mode,said vision system displays an exterior scene around the vehicle;wherein the displayed exterior scene is around the vehicle and has afirst limited extension in a first dimension and a second limitedextension in a second dimension, and wherein said second dimension isgenerally orthogonal to said first dimension, and wherein the result ofa tangent steradian accuracy of said at least one inclination sensordivided by a longer one of said first and second limited extensions isgreater than the highest resolution of at least one of said imagingsensors having its field of view along said longer limited extension'spath.
 2. The vision system of claim 1, wherein said processing systemprocesses outputs of said inclination sensors of said plurality ofimaging sensors to determine a misalignment of one of said plurality ofimaging sensors relative to the vehicle.
 3. The vision system of claim1, wherein said at least one other inclination sensor comprises aninclination sensor of one of said plurality of imaging sensors.
 4. Thevision system of claim 1, wherein said at least one other inclinationsensor comprises a plurality of inclination sensors.
 5. The visionsystem of claim 1, wherein said at least one other inclination sensorcomprises an inclination sensor of the vehicle.
 6. A vision system for avehicle, said vision system comprising: a plurality of cameras disposedat the vehicle and having respective exterior fields of view, saidcameras capturing image data; wherein each of said cameras includes acamera inclination sensor; at least one vehicle inclination sensordisposed at the vehicle; a processing system that is operable to processoutputs of said camera inclination sensors and said vehicle inclinationsensor to determine an alignment or misalignment of said cameras at thevehicle; a display screen in the vehicle and viewable by the driver ofthe vehicle, wherein said vision system is operable to display imagesrepresentative of captured image data on said display screen; whereinsaid vision system is operable in a virtual top view vision mode, andwherein, when operating in said virtual top view vision mode, saidprocessing system processes captured image data to at least one ofcombine captured image data and manipulate captured image data, andwherein, when operating in said virtual top view vision mode, saidvision system displays an exterior scene around the vehicle on saiddisplay screen; and wherein the displayed exterior scene is around thevehicle and has a first limited extension in a first dimension and asecond limited extension in a second dimension, and wherein said seconddimension is generally orthogonal to said first dimension, and whereinthe result of a tangent steradian accuracy of said camera inclinationsensor divided by a longer one of said first and second limitedextensions is greater than the highest resolution of said camera havingits field of view along said longer limited extension's path.
 7. Thevision system of claim 6, wherein said processing system is operable tocompare an output of each of said camera inclination sensors with anoutput of said vehicle inclination sensor to determine a degree ofmisalignment of each of said cameras at the vehicle.
 8. The visionsystem of claim 7, wherein, responsive to said comparison of outputsresulting in a determination of a misalignment of at least one of saidcameras, said vision system is operable to adjust one of (i) said atleast one of said cameras and (ii) processing of image data captured bysaid at least one of said cameras.
 9. The vision system of claim 6,wherein said camera inclination sensors and said vehicle inclinationsensor each comprise a multi axis inclination sensor, and wherein saidprocessing system can determine at least two of (i) a roll of thesensor, (ii) a pitch of the sensor and (iii) a yaw of the sensor.
 10. Avision system for a vehicle, said vision system comprising: a pluralityof cameras disposed at the vehicle and having respective exterior fieldsof view, said cameras capturing image data; a camera inclination sensorat each of said plurality of cameras and used to determine anorientation of the respective cameras relative to the earth, whereinsaid camera inclination sensors are fixedly disposed relative to therespective cameras; at least one vehicle inclination sensor fixedlydisposed at the vehicle and used to determine an orientation of thevehicle relative to the earth; wherein said camera inclination sensorsand said vehicle inclination sensor comprise multi axis inclinationsensors; wherein said processing system is operable to process outputsof said camera inclination sensors and said vehicle inclination sensor;wherein said processing system is operable to determine at least two of(i) a roll of each of said inclination sensors, (ii) a pitch of each ofsaid inclination sensors and (iii) a yaw of each of said inclinationsensors; wherein said processing system compares an output of each ofsaid camera inclination sensors with an output of said vehicleinclination sensor to determine a degree of misalignment of each of saidcameras relative to the vehicle; a display screen in the vehicle andviewable by the driver of the vehicle, wherein said vision system isoperable to display images representative of captured image data on saiddisplay screen; wherein said vision system is operable in a virtual topview vision mode, and wherein, when operating in said virtual top viewvision mode, said processing system processes captured image data to atleast one of combine captured image data and manipulate captured imagedata, and wherein, when operating in said virtual top view vision mode,said vision system displays an exterior scene around the vehicle on saiddisplay screen; and wherein the displayed exterior scene is around thevehicle and has a first limited extension in a first dimension and asecond limited extension in a second dimension, and wherein said seconddimension is generally orthogonal to said first dimension, and whereinthe result of a tangent steradian accuracy of said camera inclinationsensor divided by a longer one of said first and second limitedextensions is greater than the highest resolution of said camera havingits field of view along said longer limited extension's path.
 11. Thevision system of claim 10, wherein, responsive to said comparison ofoutputs resulting in a determination of a misalignment of at least oneof said cameras, said vision system is operable to adjust one of (i)said at least one of said cameras and (ii) processing of image datacaptured by said at least one of said cameras.
 12. The vision system ofclaim 10, wherein said processing system is operable to determine (i) aroll of each of said inclination sensors, (ii) a pitch of each of saidinclination sensors and (iii) a yaw of each of said inclination sensors.